| VECTOR | [3-0-0:3] |
|---|---|
| DESCRIPTION | This course introduces the fundamentals of information theory and its applications in machine learning and statistics. Beyond compression and transmission, information theory also addresses information extraction and learning. By formulating operational problems with elegant mathematics and solving them through powerful techniques, it characterizes fundamental performance limits and offers deep insights into system design. We will cover both classical and modern topics. The course begins with core information measures—entropy, f-divergences, convex duality, and variational characterizations—followed by data compression, channel coding, and strong data-processing inequalities, which illustrate the operational meaning of these definitions. We then explore information-theoretic treatments of inference, hypothesis testing, and large deviations. Applications include deriving generalization bounds, and inequalities in machine learning, as well as insights for statistical estimation. By the end of the course, students will gain both the tools and intuition to analyze fundamental limits and to apply information-theoretic methods to modern problems in learning and inference. |
| Section | Date & Time | Room | Instructor | Quota | Enrol | Avail | Wait | Remarks |
|---|---|---|---|---|---|---|---|---|
| L01 (6747) | Th 01:30PM - 04:20PM | Rm 222, W1 | HE, Haiyun | 40 | 30 | 10 | 0 |